Adaptive relative bundle adjustment

It is well known that bundle adjustment is the optimal non-linear least-squares formulation of the simultaneous localization and mapping problem, in that its maximum likelihood form matches the definition of the Cramer Rao Lower Bound. Unfortunately, computing the ML solution is often prohibitively expensive – this is especially true during loop closures, which often necessitate adjusting all parameters in a loop. In this paper we note that it is precisely the choice of a single privileged coordinate frame that makes bundle adjustment costly, and that this expense can be avoided by adopting a completely relative approach. We derive a new relative bundle adjustment, which instead of optimizing in a single Euclidean space, works in a metric-space defined by a connected Riemannian manifold. Using an adaptive optimization strategy, we show experimentally that it is possible to solve for the full ML solution incrementally in constant time – even at loop closure. Our system also operates online in real-time using stereo data, with fast appearance-based loop closure detection. We show results for sequences of 23k frames over 1.08km that indicate the accuracy of the approach.